The present invention relates to digital image processing systems for rendering and displaying views of three-dimensional (3D) geometric models, and is particularly directed to a computer-resident mechanism that enables a system user to view images generated from 3D geometric information, and to use a convenient user interface such as a joystick and a mouse device, to navigate within animated perspective views of the image as displayed on a high-resolution raster display.
Conventional image processing systems for generating and displaying three-dimensional models customarily offer the user the ability to conduct a limited degree of navigation through a scene, including walk/drive/fly navigation with a mouse, or through the use of a joystick device. In such applications, navigation is typically constrained to predefined locations, and may or may not employ an interpolation technique to provide smooth motion from a current location to a new location. For an illustration of literature relating to model display and image processing technology, attention may be directed to the Oka et al, U.S. Pat. No. 4,600,200; Pica, U.S. Pat. No. 4,631,691; Andrews et al, U.S. Pat. No. 4,646,075; Bunker et al, U.S. Pat. No. 4,727,365; Mackinlay et al, U.S. Pat. No. 5,276,785; Amano al, U.S. Pat. No. 5,287,093; Robertson al, U.S. Pat. No. 5,359,703; Robertson, U.S. Pat. No. 5,608,850; Robertson, U.S. Pat. No. 5,689,628; and Marrin et al, U.S. Pat. No. 5,808,613.
In accordance with the present invention, the constrained capabilities of conventional 3D model display systems, such as those referenced above, are effectively obviated by a new and improved versatility digital image processing system, that is effective to controllably render and display multiple aspect and size views of a three-dimensional (3D) geometric model, such as, but not limited to an urban scene, one of which is a map or xe2x80x98bird""s eyexe2x80x99 view of the scene, and the other of which displays a relatively close, or xe2x80x98in-scenexe2x80x99 view of the 3D model image. By the use of a user interface, such as a joystick or mouse, the user is not only able to toggle between which of the two scene perspectives is displayed as a map view and which is displayed up close for in-scene navigation, but may readily navigate within the virtual world of the in-scene view of the displayed 3D image.
For this purpose, the image processing system architecture of the present invention includes a host computer having an associated high resolution display, and one or more associated user interface devices (e.g., mouse and/or joystick), through which the user controls manipulation of and navigation through images generated on the display. To facilitate manipulation of and navigation through a 3D model, such as xe2x80x98street levelxe2x80x99 movement through an urban scene, the viewing screen of the display is divided into two viewports, which display respectively different magnification views of an image of the 3D model, from points in three-dimensional space that are respectively xe2x80x98down insidexe2x80x99 and xe2x80x98away fromxe2x80x99 the scene.
A first or main viewport may comprise a relatively large region of the display screen, while a second or inset viewport may comprise a relatively small region that overlaps or is superimposed on the main viewport portion. As noted above, one of the viewports will display a map or xe2x80x98bird""s eyexe2x80x99 view of the scene, while the other viewport will display a relatively close, or xe2x80x98in-scenexe2x80x99 view of the 3D model image, and the user may toggle between the two scene perspectives. Operation of the user interface produces input parameters that control navigation within the in-scene view of the displayed 3D image.
The user may change the 3D location and orientation of a user virtual representation icon. The user icon is a software generated object that is superimposed on the displayed image to represent the user""s location and orientation within the displayed 3D model. As the user manipulates the icon, an interpolation mechanism is accessed to supply control information to a xe2x80x98view cameraxe2x80x99 operator, outputs of which are employed to create the 3D scene and to render the view, as the view is coupled to a viewport, which defines the overview and in-scene presentations of the image to the respective viewports.
The overview view of a geographic scene is preferably displayed in a fixed (e.g., North-up) orientation, and is rendered as a 3D perspective map view of the virtual world. Navigation within this xe2x80x98far awayxe2x80x99 plan view is generally limited, and operation of the user interface moves the user location horizontally and vertically in the display, or at a constant altitude within the image. The overview image maintains the user location in the center of the viewport, so that changing the user location will cause the overview image to pan in the direction opposite the user""s motion. The user may change the altitude of the overview viewpoint, to provide a magnification and demagnification action, which essentially allows the user to xe2x80x98zoom intoxe2x80x99 or xe2x80x98zoom outxe2x80x99 of the scene.
The in-scene view is a less constrained 3D perspective view, and directly represents what would be seen by a user located in the virtual world of the 3D model. The user""s line-of-sight or viewing direction may be directed at, or close to, horizontal, or it may inclined upwardly or downwardly at any elevation angle, and may face any azimuth angle (compass direction). This closely approximates how an upright human is able to view his/her surroundings within the 3D scene. The in-scene view will be typically from near-ground-level, as in the case of an urban street scene for example, but is not constrained to be so. It may be from any aspect to which the user navigates.
There are two modes of navigation that the user can perform in the in-scene view: 1) joystick mode and 2) mouse mode. In joystick mode, the user moves the joystick handle and operates buttons on the handle to control user location within the virtual 3D world. In mouse mode, the user clicks on a new vantage or xe2x80x98look-fromxe2x80x99 point, and then an appropriate target or xe2x80x98look-atxe2x80x99 point. The look-at point is employed to orient the user as though standing at the new vantage point. To select a new vantage point, the mouse pointer is placed within the main viewport, and the left mouse button is actuated. The new look-at point is selected while the user continues to hold the left mouse button down, and moves the mouse pointer. In both of these steps the mouse pointer is positioned on the actual 3D model displayed in the main viewport.
In the course of navigating through or otherwise manipulating the image, an interpolation operator is used to effect a gradual or smooth transition of the image between its starting and target locations. In addition, each viewing camera location/orientation is updated by the interpolation function assigned to it. Different interpolation functions may be assigned to a camera at different times.
As pointed out above, the user may operate a pushbutton on the user interface to xe2x80x98togglexe2x80x99 between the two scene perspectives displayed by the respective main and inset viewports. The two views are mutually synchronized, so that the viewing cameras move in a coordinated fashion to swap view locations, as the two viewports exchange their respective display roles. The user may toggle between the two views at any time, even during the transition initiated by a previous toggle state change. In this latter case, the motion simply reverses and the viewpoints begin returning to their immediately previous locations. Since the toggling mechanism of the invention cause the two views to be swapped simply and smoothly, user distraction is minimized, so as to facilitate the user retaining a mental picture of his/her location in the virtual world.
A principal benefit of this feature of the invention is that it allows a user at street level in an urban scene to readily and easily ascend or rise to a higher vantage point, in order to realize a better sense of location within the scene. This permits the user to ascend fully to the overview map viewpoint, or to rise up just far enough to realize where he/she is, and then immediately return to street level of the virtual 3D image. This feature also provides a convenient mix of close-up, on-the-street navigation with longer-range navigation. The user on-the-street can xe2x80x98pop upxe2x80x99 to the overview viewpoint, roam quickly to a distant part of the scene, and then descend back to street level in the new location. It additionally avoids the problem of having to provide appropriate scale factors to a single navigation model for both close-up and long-range motions, and avoids the distraction of forcing the user to decide when to use one or the other.
In addition to swapping views, the user may navigate through a view, using either the joystick or mouse to manipulate the user""s location and orientation within the scene. Advantages of this user icon transition mechanism include the fact that only a single press/drag/release mouse operation is required to position the user icon at a predetermined location (e.g., height above a point on a surface) within the 3D scene, orient the user icon, set the viewing direction, and set the look-at point to another point on a target surface within the scene. The user""s viewpoint will not change within the window until the operation is complete. This allows the other view to present a preview of the user""s new viewpoint. Moreover, as will be described, this same interaction and visual representation can be employed for mensuration.
Pursuant to a further feature, the invention provides a reduced complexity user icon having a visual representation that conveys several pieces of information to the user. A first is the location of the user""s head, or viewpoint in the virtual world. In a non-limiting but preferred implementation, the user icon""s head may be represented by a colored (e.g., red) sphere. Second, the heading or line-of-sight of the user corresponds to the xe2x80x98forwardxe2x80x99 direction faced by the user toward a look-at point in his/her local frame of reference.
The look-at point may be represented by a (yellow) cube whose center is placed at the looked-at point in the 3D scene. The look-at point also determines the center of the base of a semi-transparent (blue) cone. Heading (view direction, or the azimuth and elevation of the view) may be indicated in the displayed image by the axis of the cone. The apex of the cone touches the surface of the sphere, and the xe2x80x98boresightxe2x80x99 axis lies along a ray from the center of the sphere through the point of intersection of the cone apex with the sphere""s surface. The user""s field of view is defined by the angle subtended by the cone. Because a cone is used, the field of view indication only approximates what is seen in the displayed rectangular viewport. For a more accurate representation a rectangular frustum whose cross section matches the aspect ratio of the viewport may be used.
In accordance with a mensuration mode aspect of the invention, the cone is not displayed. Instead, a line or very narrow cylinder may be used to connect the user icon head and the look-at point. The distance to the look-at point being viewed corresponds to the height of the cone along the line-of-sight axis from the center of the sphere to the base of the cone. The tallness of the user may be represented by a thin cylinder or leg whose axis is vertical and whose upper end terminates at the center of the sphere. The base of the leg corresponds to the observer""s foot location (on the ground). This facilitates measurements by placing the user icon on a surface such as the ground, to determine the vertical offset of the viewpoint above the surface.
The distance between the user icon foot and the icon head may be made adjustable, rather than fixed. The user icon can then be used to approximate human viewing characteristics, by placing the icon in a standing position on a surface in the scene (e.g. on the sidewalk of a city street), and looking outward from some distance above the surface. The variable height of the user icon does not affect its ability to be translated up or down any amount in the scene.
As an alternative shape the features of the user icon may be incorporated into a more xe2x80x98humanxe2x80x99-looking iconic representation, to provide the additional benefit of a familiar scale of reference, that the human user could employ as an additional visual cue, to aid in estimating the sizes of objects within the scene. The user icon may also be employed for navigation under mouse control and for mensuration.
When conducting mensuration, dimensions of features within the image viewed by the user in the main viewport can be measured. In this mode, moving the user location/orientation interactively allows the user to select respective proxy vantage and look-at points. During the selection process, a mensuration icon is displayed between the proxy vantage and look-at points. In addition, the display may show parametric values associated with distance between the proxy vantage and look-at points, change in horizontal distance, and change in vertical distance next to the mouse pointer. Upon completion of the proxy viewpoint specification, the user""s viewpoint does not change, as when moving to a new location. This allows the user to measure additional features within the displayed 3D image.
Advantages of the mensuration feature of the invention include the use of a only single press/drag/release mouse operation for measuring any feature that can be seen within the 3D scene from the main viewport. Displaying a mensuration icon provides the user with visual feedback of what is being measured, along with distance information aligned with the mouse pointer. This enables the user to focus on the mensuration technique, without having to look elsewhere for distance information. A same mensuration menu item may be selected to place the user back into the normal viewpoint selection mode.
In accordance with an additional load management feature of the invention, objects may be selectively introduced into or removed from the displayed scene. This feature may be employed, for example, if an object occludes a prescribed portion of the scene, or impedes a user""s understanding important aspects of the scene. Pursuant to the invention, and object (such as a building or a city block of an urban scene) may be xe2x80x98grownxe2x80x99 from a (ground) surface of the scene, or it may be xe2x80x98translatedxe2x80x99 vertically from a location beneath the ground surface into the scene. A model unit may be also be gradually animated out of or xe2x80x98unloadedxe2x80x99 from the scene, by reversing the steps through which it was previously introduced. The geometry of the unloaded model unit need not be discarded; instead, it may simply be made invisible until it is to be redisplayed.